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Motivation 


Motivation 


When you take a picture with a modern digital camera, it can be seen that 
they cannot yet match the eye's ability to manage contrast. An image sensor 
or display medium's contrast ratio defines the distance between the darkest 

black and the lightest white that the device records or displays. The eye's 

contrast ratio of 1:100,000 is 24 times greater than the 1:4096 of a typical 
digital camera. As a result, even in correctly exposed photographs, shadows 
are often too dark and highlights too bright, creating a noticeable loss of 


detail. 


Humans can discern very high range of brightness values. Photography is 
limited to a much lower dynamic range. 


picture 


high dynamic range 


Figure: High dynamic vs. low dynamic range 


Figure: From left to right: underexposed, correct exposure, overexposed 


We can, however, compute a high dynamic range (HDR) picture from 
multiple exposures. Because we cannot display an HDR picture due to 
limitation, we will use tone mapping to compress contrast intro a 
displayable range. 


Introduction 


Introduction 


High dynamic range (HDR) digital imaging systems increase visual fidelity 
by integrating contrast ratio with observer-based models of color perception 
across multiple exposures of a single scene. They use mathematical 
transformations known as tone mapping operators to display HDR images 
on low dynamic range (LDR) devices such as monitors. 


We will uses a series of photographs of a scene to produce an image that is 
as Close as possible to what the human eye would see. To turn a photograph 
into an HDR image, we first extract the radiance information from the 
photographs and then tone-map it into a single LDR image. Both the 
reconstruction and tone mapping algorithms rely on matrix operations so 
we will implement in MATLAB. 


Background 


Background 


HDR increases visual fidelity by integrating contrast ratio with observer- 
based models of color perception. Relative values of real scene radiances 
are rarely captured by cameras because of non-linearities and clipping. The 
response curve of a camera maps the real scene radiance to the digitized 
pixel values. HDR methods use response curve to find real scene radiances. 
Tone mapping filters such images for display on low dynamic range (LDR) 
ae 
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Radiance Mapping 


The LDR of a single digital photograph contains accurate radiance 
information only for correctly exposed pixels. Changing the exposure 
captures radiance information for different sets of pixels. To accurately 
analyze a sequence of photographs to extract radiance information for the 
entire scene, we must take into account the way the camera sensor responds 
to different levels of light. First, we try and get the camera response curve f, 
which tells us how scene radiance E is mapped to pixel brightness Z. Using 
the inverse of f allows us to reproduce actual scene radiance E. The curve f 
is different for each camera and we compute f from a series of exposures. 


Figure: Exposure series used to compute camera response curve 
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Figure: Recovered response curve f 


Implementation: HDR Method 
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From Figure 1: Image Acquisition Pipeline, we could see how the scene 
radiance(X) is non-linearly mapped into its final digital values(Z). In other 
words, f(X)=Z, the composition of the characteristic curve of a device and 
all the nonlinear mapping, is a nonlinear function. Non-linearity is mostly 
introduced by later processing steps such as analog to digital conversion 
and remapping. Therefore, our method to construct a High Dynamic 
Range(HDR) picture is to first recover the response curve, f(X)=Z and then 
by using the pixel values from the series of (LDR) pictures, we’re able to 
get the scene radiance, which is like a reversed process. Finally, our HDR 
picture will be constructed from these values of scene radiance. 


Our general process will be: 
1. We’ll take pictures with different exposure times. 


a) Knowing the exposure X and the exposure time delta t, we’re able to 
recover irradiance through formula E = X/delta t. 


b) Since f(X)=Z and function f could be reasonably conceived as increasing 
function, its inverse function is well defined. Then, we have 


= f 


c) Using the formulas in a) and b), we have 


Aye = FUE At) 
where i is the index of sample(pixel) and j is the index of pictures with 
different exposure times. The formula also equals to 


f  (Zi;) = FB, At; 


taking natural log on both sides, we will have 


Inf *(Zi;) =n EB; +n At; 


Making X=g(Z) the inverse function of Z=f(X), we get 


g( Z;;) = n EB; +1n At; 


d) Thus, we transform our problem into a problem minimizing least-squared 
error between g(Z) and In(E) and In(delta t), which could be summarized by 
the following formula: 
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Note that the second term here is for smoothing g(Z). And lambda is a 
scaling factor subject to different pictures. 


3. After we get the response curve by solving the equation above, when we 
use the pixel values Z of those LDR pictures, we’ll be able to get the values 
of irradiance X at each point of the scene. In other words, we’re trying to 
figure out in different areas, which group of pixel values from different 
LDR pictures with different exposure times is closest to the corresponding 
irradiance and best to reveal details of that area. Thus, it enables us to 
construct HDR picture in which pixel values in different areas are extracted 
from the LDR picture that captures its details best. 


4. We finish our task to construct a HDR picture here. However, we still 
have to solve another problem, which is to display a HDR picture on a LDR 
display. And this leads to the second part of our implementation: Tone- 


mapping. 


Implementation: Tone Mapping Method 


Tone Mapping 


Tone Mapping is a tool that allows us to map a High Dynamic Range 
(HDR) image for display on a Low Dynamic Range (LDR) medium, such 
as an LCD screen. We have implemented Reinhard's algorithms in doing 
this, which revolve around the idea of scaling the pixel values of the image 
by a factor to fit in the LDR. In Reinhard's paper, two implementations are 
found. Let us call these two: Initial Scaling, and Dodge-and-Burning 
(DAB). 


For both implementations, we compress the 3 different channels of Red, 
Green, and Blue into luminance values by: 
Equation: 


Lw = (0.2125)R + (0.7154)G + (0.0721)B; 


We denote the luminance value by L,, and Red, Green, Blue by R, G, B 
respectively. 


As we would like to avoid singularities, we adjust all luminance values for 
each pixel with a small value, e. 


For the Initial Scaling scheme, we first obtain the key value of the scene by 
computing: 
Equation: 


key = area (>: (> (log (del + tw) 


x 7] 


where we sum over all x and y values indicating pixel location. Now, we 

normalize all pixel values with respect to the middle-grey of the scene. The 
middle-grey is the subjective middle brightness region of the scene and for 
images of normal brightness, this middle-grey value typically maps to 0.18, 


and always lies in between 0 and 1. Once we have normalized, we scale all 
pixel values by to get a LDR image: 
Equation: 


L(z,y) 


La(z,y) = Te Desay 


In the DAB scheme, we obtain two Gaussian functions of different sizes 
located concentric with each other, where the ratio of variances is set to 1.6 
(this value can be changed slightly to meet better quality tone mapping). We 
convolute the image with both Gaussian functions and calculate: 

Equation: 


Vi (2, Yy, s) ~~ V2 (x,y, 8) 


V iY; = 
ED ayaa (ays) 


(1) 


where V; and V2 are the Gaussian functions of different sizes, y is a 
parameter (=8 in our case), a is the subjective middle-grey value of our 
scene and s is the scaling ratio of the Gaussian functions. 


In doing so, we hope to find a small outcome that is less than or equal to a 
tolerance level that we give (in our case 0.05). If this value meets the 
tolerance level, we try to enlarge the variance (and by doing so enlarge both 
gaussians) so that we can compute (1) over again to re-evaluate if this larger 
area still meets the tolerance. We keep doing this until our Gaussians cover 
the entire image or we meet the largest area that is less than the tolerance 
level. We then keep the convoluted values of the image with the smaller 
Gaussian function and use this for our new LDR image. 


What we are essentially doing in this scheme is finding the smallest contrast 
difference obtainable in the largest area and keeping these values. In effect, 
we are smoothing the image to fit the LDR yet keeping the edges and 
details of the image. 


In these two ways, we are able to obtain a LDR from a HDR image. 


Results 


Results 


In our experiment, we take three groups of LDR pictures and try to 
construct three HDR pictures from them. For each group of these pictures, 
we take 6 LDR pictures with different exposure times. We’|l display the 
response curve for each color channel: red, blue and green and our final 
HDR picture. 


Group 1 


Response Function Group 1 Red 
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Response Function for Channel Red 


Pixel Value z 


Response Function Group 1 Green 
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Response Function for Channel Green 


pixel value z 


Response Function Group 1 Blue 
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Response Function for Channel Blue 


LDR picture with exposure time 1/50s 


LDR picture with exposure time 1/125s 


LDR picture with exposure time 1/320s 


LDR picture with exposure time 1/800s 


LDR picture with exposure time 1/1600s 


LDR picture with exposure time 1/4000s 
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Final HDR picture for Group 1 


Comment: It’s impressive to see the details both of the light and of the 
clothes. For these kind of pictures, it’s always easy to lose the details of the 
light since the pixel values of the area around the light will get saturated. 
But, through HDR, we could keep the details of the light without losing any 
other details. 


Group 2 


pixel value z 


Response Function Group 2 Blue 


log exposure 


Response function for Channel Blue 


Pixel alue z 


Response Function Group 2 Green 


log exposure 


Response function for Channel Green 


pixel value z 


Response Function Group 2 Red 


log exposure 


Response function for Channel Red 


LDR Picture with exposure time 1/50s 


LDR Picture with exposure time 1/100s 


LDR Picture with exposure time 1/400s 


LDR Picture with exposure time 1/1000s 


LDR Picture with exposure time 1/2000s 


LDR Picture with exposure time 1/4000s 


Final HDR picture for Group 2 


Comment: Once again, we almost see the details of the lamb. Although it’s 
distorted a bit, we know there’s something in it, instead of just completely 
white for the whole area. What’s amazing is that we could clearly see 
what’s going on in the background. All the details of the bags are clearly 
revealed. 


Group 3 


pixel value z 


Response Function Group 3 Blue 
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Response Function for Channel Blue 
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Response Function Group 3 Green 


log exposure 


Response Function for Channel Green 


pixel value z 


Response Function Group 3 Red 


log exposure 


Response Function for Channel Red 


LDR Picture with exposure time 1/160s 


LDR Picture with exposure time 1/320s 


LDR Picture with exposure time 1/640s 


LDR Picture with exposure time 1/1250s 


Final HDR Picture for Group 3 


Comment: Once again, we could see a lot of details both in over-exposed 
area and in under-exposed area. For example, we could see some words on 
the white container on the right hand side of the lamb. It’s impossible for us 
to see those words when we take a picture with relatively large exposure 
time. Similarly, we could see the details in the dark area. For example, we 
could see that there’s a purple/blue stuff lying on the black bag. However, 
it’s likely that we’ll barely see it when the exposure time of the picture is 
relatively small. 


Conclusion 


Conclusion 


Through this project, we’re able to use several Low Dynamic Range(LDR) 
pictures and their exposure times to create a High Dynamic Range(HDR) 
picture and display it successfully on LDR display. 


Limitations and Future Work 


Our project has three major limitations. First of all, since there are 
parameters in the HDR and Tone Mapping algorithms such as lambda, the 
scaling factor of the smoothness term in equation() are subjective to 
different pictures,so it’s hard for us to figure out the parameters. Every time 
when we input a different series of LDR pictures, we have to readjust the 
parameters. Therefore, in the future, we hope to come up with an algorithm 
that could calculate the parameters according to our input pictures. 


The second limitation happens when we take a series of pictures with 
different exposure times, so we have to ensure that everything in the scene 
has to stay in the same position. Otherwise, the HDR picture will get 
blurred. It gets harder when we increase the exposure time, since we’re 
exposed more to hand shaking and the changes of environment. For this, we 
come up with two solutions for this limitation. On one hand, we can 
“integrate” the HDR algorithm into hardware. For example, we can build an 
app in Android so phones are able to take several pictures by themselves 
and thus our pictures are less vulnerable toward vibration. On the other 
hand, we can build another algorithm which could pre-process our pictures 
to ensure that everything stays in the same position. Additionally, for over- 
exposure pictures, we could increase the [SO(photosensitivity of sensors) to 
decrease the exposure time. 


Our third limitation comes from assumption of our algorithm. When we 
take those LDR pictures, we assume that the irradiance of each point 
doesn’t change. In other words, since we’re doing 1-1 mapping(one 
exposure time to one irradiance), if we change the lighting of environment, 
it will mess up with the recovering of device’s response curve. Therefore, 


we’re unable to add radiance from camera flash. This will largely reduce 
HDR’s real application. We hope to come up with a method to deal with 
this case. 


Finally, we hope to extend our algorithm so that it will be able to 
manipulate other camera variables, such as aperture size and ISO as we 
mentioned above. These algorithms will allow us to have more degree of 
freedom on our control of pictures. 
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INTRODUCTION 


*An image sensor medium’s contrast ratio defines the distance between 
the darkest black and the lightest white that a camera displays. 


‘The eye's contrast ratio of 1:100,000 is roughly 24 times greater than 
the 1:4096 of a typical digital camera. 


*As a result, even in correctly exposed photographs, shadows are too 
dark or highlights are too bright, producing a noticeable loss of detail. 


‘A High Dynamic Range (HDR) digital imaging system uses a series of 


photographs from one scene to produce an image that is close to what 
the human eye would see 


MISSION STATEMENT 


To construct high dynamic range images from differently exposed 
photos and display them on low dynamic range displays. 


BACKGROUND 


+HDR increases visual fidelity by integrating contrast ratio with observer- 
based models of color perception 


‘Relative values of real scene radiances are rarely captured by camera - 
nonlinearities, clipping 


“The response curve of a camera maps the real scene radiance to the 
digitized pixel values 


*HDR methods use response curve to find real scene radiances. Tone 
Mapping filters such images for display on low dynamic range (LDR) 
displays 
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Figure 1 - From real scene radiance to digitized value — Illustrates non linearities 
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= Figure 2 - Tone mapping compresses HDR 
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Images with various exposures 


Response Curves for RGB (Left), HDR Image (Right) Picture displayed on LDR displays 


{Pictures with different exposure time 


1,.HDR radiance map 
2.Recover response curve of the camera by minimizing: 


E 2.Convolute with two Gaussian filters 
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oe Values at each point from response curve 4, Get the HDR image which could be displayed on LDR displays 


4, Construct HDR radiance map using those recovered values 
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CONCLUSIONS 


In our process we have done the following: 

+ Acquired exposure time dependent images 

+ Produced HDR images using Debevec's method 

+ Converted HDR to LDR for viewing on LDR displays by 
Reinhard’s method 


Limitations: 

+ Parameters for HDR and Tone Mapping algorithm are image 
based and subjective 

+ Computational power and time dependent photo resolution on 
HDR and Tone Mapping 


Figure - Response Curve (Green) Top 
Right Image 


Future Work: 

+ Adaptation of HDRI code with manipulation of other camera 
variables 

«Use of HDRI algorithm with added radiance from camera flash 
+ Computationally cheap algorithms or methods 
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